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In the IDEA Student Ratings of Instruction (SRI) system, 
students complete self-ratings of their work habits, 
motivation, and background preparation. Average 
course ratings on these characteristics are important 
variables used in adjusting scores on student ratings of 
progress on relevant objectives (PRO), excellence of the 
instructor, and excellence of the course (Benton, Li, 
Brown, Guo, & Sullivan, 2015). The purpose of this 
study was to examine whether student self-ratings are 
related to the instructor's corresponding ratings of the 
class as a whole. 


Method 

Data Source 

Analyses were performed on data collected through the 
IDEA Legacy SRI platform from 2002 to 2016. To 
increase reliability, we restricted the analytic sample to 
courses with a minimum of 10 responses. To reduce 
bias introduced by courses with low response rates, we 
included only classes with a response rate of at least 
50%. Course-level mean score ratings were included 
for student responses to items on the Diagnostic Form 
(DF) and Short Form (SF), if the instructor responded to 
the respective student characteristic items on the 
Faculty Information Form (FIF). As a result, 242,538 
average course ratings were included from the DF and 
102,919 were included from the SF. 


Measures 

Students rated themselves on the following three 
items, using a five-point scale (1 = Definitely false, 2 = 
More false than true, 3 = In between, 4 = More true 
than false, 5 = Definitely true): 


"As a rule, | put forth more effort than other students on 
academic work." (work habits) 


"| really wanted to take this course regardless of who 
taught it." (motivation) 


"My background prepared me well for this course’s 
requirements." (background preparation) 


Instructors rated the class as a whole on the following 
characteristics, choosing one of four possible 
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responses (Had a positive impact on learning, Neither 
a positive nor a negative impact, Had a negative 
impact on learning, Can't judge): 


"Student effort to learn" (work habits) 
"Student enthusiasm in the course" (motivation) 


"Adequacy of students' background and preparation for 
the course" (background preparation) 


Hypothesis and Analyses 

We hypothesized that average student self-ratings on 
work habits, motivation, and background preparation 
would be higher in classes where the instructor 
indicated the respective characteristic had a positive 
impact on learning than a negative impact. To test this 
hypothesis, we performed independent t-tests where 
the dependent variable was the respective average 
student self-rating and the independent variable was 
the instructor group (positive- vs. negative-impact). Due 
to the large sample sizes, type | error was set at .001. 
We employed Cohen's d (1992) as a measure of effect 
size, where small = 0.20, medium = 0.50, and large = 
0.80. 


Results and Discussion 

Table 1 presents results of the t-tests and Cohen's d 
computations. Average student self-ratings were 
significantly higher on all three dependent variables in 
classes where the instructor indicated the respective 
characteristic had a positive impact on learning than in 
classes where it had a negative impact. A strong effect 
was found for student motivation (d = 0.76), whereas 
background preparation (0.57) and work habits (0.28) 
showed medium and small effects, respectively. 


The current findings provide evidence of the concurrent 
validity of student self-ratings on work habits, 
motivation, and background preparation. Significant 
and meaningful relationships exist between average 
student self-perceptions on these characteristics and 
the instructor's overall impressions of the students 
enrolled in the course. 
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Table 1 


Summary of Means, Standard Deviations, and t-tests for Student Self-ratings by Instructor-reported Impact of the Student Characteristic 


Instructor-reported positive impact Instructor-reported negative impact 
Student ratings M SD n M SD n t p Cohen’s d 
Work habits 3.97 0.31. 196,448 3.88 0.30 44,609 56.27 < .001 0.28 
Motivation 3.70 0.54 186,977 3.29 0.55 45,214 143.80 < .001 0.76 
Background 3.97 0.42 39,370 3.73 0.44 17,591 61.17 < .001 0.57 
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